A Novel Virtual Emergency Medicine Residents-as-Teachers (RAT) Curriculum

Audience The Residents-as-Teachers (RAT) curriculum is designed for emergency medicine (EM) residents of all years (PGY1–4). Length of Curriculum The curriculum is divided into three hour-long sessions. The entire curriculum can be run as a single block or can be spread out over multiple days. Introduction The Accreditation Council of Graduate Medical Education (ACGME) and the Liaison Committee on Medical Education (LCME) both require residents to receive training in teaching medical students and junior residents. They also require opportunities for residents to participate in teaching and be assessed on their effectiveness in this role.1,2 However, the ACGME does not provide guidance or require formal curricula on molding residents into effective teachers. Many programs and institutions have incorporated RAT curricula as a solution to provide residents with the skills necessary to create an excellent educational environment for junior learners. These curricula have been embraced by many specialties, including Emergency Medicine (EM).3–6 The effectiveness of the teaching received during the clinical rotations has important long-term effects on medical students, and may impact their future career choices in medicine.6 The COVID-19 pandemic has also required education institutions to vastly alter the delivery of their didactics, including moving to a virtual platform. A completely online format has many benefits that extend beyond the pandemic, such as easier access to participants (including those off-service or at remote sites), no requirement for a physical space, and easier recording of sessions. Educational Goals To provide residents with an introduction to teaching techniques that can be utilized on-shift to facilitate an excellent educational experience for junior learners while balancing the resident’s patient care responsibilities. Educational Methods The educational strategies used in this curriculum include PowerPoint (Redmond, WA) slideshows given by a live presenter via the telecommunications platform Zoom (San Jose, CA), viewing of videos demonstrating curriculum topics, simulation-based learning through role-play, and small-group discussions including simulation debriefing. Research Methods A survey was distributed to residents before and following the completion of the three training sessions to assess resident satisfaction with the delivery of the content and comfort with the teaching tools discussed. Suggestions on potential improvements were also assessed to inform changes to future iterations of the curriculum. Comfort regarding the included teaching tools was assessed using a five-point Likert scale. After completion of the curriculum, rotating medical students were provided with an evaluation form to assess if residents were teaching using the techniques from the course. Results Both the pre-curriculum and post-curriculum surveys had a response rate of 61.1%. Student’s t-test showed a statistically significant increase in mean resident comfort level with the teaching strategies post-curriculum (3.05 to 3.83, p < 0.01). Medical student evaluations have shown, overall, that the majority of residents are utilizing the education techniques on-shift. There were no significant differences found in medical student perception of resident use of taught skills between those who had and had not attended the sessions. However, all but one assessed skill showed higher utilization in those who had attended the correlating session. Discussion The educational content was effective in improving the residents’ comfort with the teaching strategies presented, and residents are utilizing these techniques on-shift. Through implementation, we discovered that presenting a curriculum over video conferencing required additional administrative support to help ensure efficacy of break-out groups. Based on resident feedback after the first session, multiple changes were made, including providing residents with hand-out references for use during the role-playing sessions. The success of this curriculum demonstrated the feasibility and utility of running a RAT curriculum entirely in a virtual format. Topics Residents-as-teachers, distance learning, role-playing, virtual curriculum, video conferencing, One Minute Preceptor, feedback, “What if?” game, Aunt Minnie, SPIT, activated demonstration, self-directed teaching tools, teaching scripts, Post-It Pearls.


USER GUIDE
The Emergency Department (ED) is a rich source of educational opportunities, but also presents many obstacles to educators, including time limitations, frequent interruptions, and lack of physical space for teaching. 10 Therefore, it is important to prepare emergency medicine (EM) residents to teach effectively while still directing efficient patient care. We present a virtual curriculum that utilizes lecture, role-playing, and debriefing to educate residents on the One Minute Preceptor, providing effective feedback and other on-shift teaching tools (eg, Aunt Minnie, Post-it Pearls, SPIT) for teaching effectively in the ED.
COVID-19 has led to many innovations in medical education due to the need for social-distancing, such as the implementation of remote curricula via online video-conferencing. 11 Sessions in which procedures and other skills are provided over a videoconferencing platform have been successful previously. 12 Therefore, due to the inability of residents to gather in person, our curriculum was designed so that it could be presented entirely over video conferencing software, such as Zoom (San Jose, CA). In light of the current global pandemic, and the increasingly important role of online educational delivery, this curriculum represents a novel way to ensure residents are receiving instruction on a critical topic in a safe and effective manner.
We believe that even after pandemic-related restrictions are lessened, the online format of this curriculum will continue to be beneficial. First, it decreases the need for a large physical space to conduct the sessions (which is increased by the need to conduct the breakout role-playing sessions). Second, it can allow individuals who are unable to attend in person, such as those on off-service or remote rotations, to participate. Finally, the online format facilitates easy recording of the didactic portions of the sessions so that they can be reviewed by those who could not attend.

Problem identification, general and targeted needs assessment:
The impetus behind our decision to devise and implement our virtual RAT curriculum was based on a restructuring of our institution's medical student curriculum to emphasize earlier exposure to clinical environments, including the ED, during the M2 year. This resulted in the expansion of teaching interactions between residents and junior medical students on-shift. There has also been a departmental focus on improving the quality of the educational experience to medical students during their EM clerkship. This prompted discussions between program leadership and residents which revealed resident concerns about their ability to provide effective and efficient teaching onshift while continuing to manage patient care.
RAT curricula have been created across different specialties and use a variety of techniques such as lecture, role-playing, simulation, and objective structured teaching exercises (OSTE). 7 Efficacy of published curricula have also been evaluated according to different levels of the Kirkpatrick's training evaluation model. 13 In a 2017 survey, 80% of programs had a RAT curriculum; 90% of these curricula included lectures and approximately half included role-playing. 14 While RAT curricula have been studied extensively, the educational clearinghouses of MedEdPORTAL and the Journal of Education and Teaching in Emergency Medicine (JETem) have very few published RAT curricula that can be rapidly adopted by other programs. [14][15][16][17][18][19][20][21] Only one of these curricula is aimed specifically at EM residents. 14 The Emergency Department is a unique teaching environment, with variable pace, acuity, and patient volume, and therefore it is important to have curricula available that address the needs of residents specifically teaching in the ED.
Unlike existing curricula, this RAT curriculum was designed to be implemented in a completely virtual format, made necessary due to medical students returning to the clinical environment while educational conferences were still required to be virtual. We felt that it was essential to maintain an interactive component, and therefore role-playing was utilized to allow residents to practice skills acquired in the lectures.
To create the ideal content for our curriculum, we reviewed the literature for high yield topics for a resident-focused intervention. Following the literature search, potential topics most important to EM residents were decided upon by the author group, which consisted of senior EM residents and residency program leadership at a single academic institution. Based on the results of this discussion, the residents were then surveyed to select the three topics that they felt would be most valuable. This curriculum was intended to enhance the medical student experience by changing resident behavior while teaching onshift. It was also intended to improve resident confidence in on shift teaching and their ability to balance their time between clinical duties and teaching responsibilities.

Goals of the curriculum:
The Residents as Teachers curriculum aimed to provide residents with tools needed to facilitate effective on-shift medical student education via an entirely virtual format. Individual sessions focused on the One Minute Preceptor teaching method, provision of high-quality feedback, and effective integration of teaching into busy clinical shifts.

Results and tips for successful implementation:
The curriculum was implemented through a series of three virtual, video-based sessions held through October-December of 2020, with approximately 8-12 resident learners available per session. 22 out of 36 residents were able to attend at least one session, with 5 residents being able to attend two sessions. Only two residents were able to attend all three sessions. The relatively low number of residents that were able to participate completely was primarily due to residents' clinical duties interfering with their ability to attend weekly didactic conference. Programs desiring to expose a greater portion of their residents to the curriculum could seek to create a small amount of protected time for their residents and/or condense the sessions into a single day. Each session consisted of an introductory lecture followed by several small group break-out sessions for role-playing scenarios with facilitated debriefings. These role-playing scenarios and debriefing sessions allowed residents to practice the concepts introduced in the lecture. The first session included a brief introduction to the overall curriculum. Subsequent sessions began with a brief overview of the preceding session to facilitate retention of knowledge through spaced repetition. The sessions were presented in the following order: One-Minute Preceptor, Feedback, and Tips and Tricks.
Evaluation of the sessions was completed using both pre-and post-curriculum surveys, which included a series of 6 questions targeting the teaching techniques emphasized by the curriculum (Figure 1). In total, residents reported a statistically significant increase in comfort with the teaching strategies when considered as a whole (3.05 to 3.83, p < 0.01). Mean comfort was increased in every question assessed, with residents reporting the most significant increase in comfort in understanding learners' response to feedback (2.55 to 3.68, p < 0.001). Resident behavior following the session was assessed through post-shift surveys distributed to second year medical students during their 2-week required EM rotations. Students were asked to respond to five questions regarding whether residents used skills taught in each of the individual sessions (discussing expectations and goals, teaching principles, providing post-shift feedback, eliciting student input on feedback, and utilizing a variety of teaching techniques) after every shift. Resident attendance of individual sessions was determined using an attendance log, and answers were compared between residents who did and did not attend the sessions using a Chi-square test for all questions except the question regarding teaching principles, which was analyzed using Fisher's Exact Test (Table  1). session. These results were limited by a small n of individual residents who were able to attend each individual teaching session, and the high amount of "yes" responses by students to each question.

Evaluation and Feedback:
After completion of the first session, both instructors and learners reported issues with the break-out groups, with instructors noting difficulties in managing splitting groups through the software while trying to lead discussions and guide the breakout groups. To address these difficulties, we decided to add an outside facilitator (in this case, our administrative assistant who was experienced with managing Zoom breakout rooms) to separate residents into groups and keep track of timing. Ideally, this individual should not be one of the session facilitators to allow them to travel between small groups unhindered to observe the role-playing activities and to stimulate discussion.
Learners further reported uncertainty about expectations and best practices during the role-playing sessions following the first session. Future sessions were adapted with the addition of a feedback video and acted scenario by facilitators for the feedback session. Hand-out materials were also provided during sessions to supplement slide-based presentations so that residents could reference the high-yield information from the lecture while completing the role-playing scenarios. The handouts were created on Google Drive (Mountain View, CA) and included links to each of the role-playing scenario scripts at the end. We initially encountered permissions issues with some of these files when residents tried to access them unsuccessfully; appropriately broad sharing settings for all files should be confirmed prior to the start of the exercise. Once this was resolved, we found Google Drive to be a platform where the residents were easily able to access all the tools they needed for the role-playing scenarios.

Pre-Session Preparation:
-It is strongly recommended that the session handouts and the scenario scripts are loaded into a GoogleDrive folder. These can then be linked to the scenario scripts into the bottom of the handouts for ease of access. -Ensure that sharing permissions are correct (eg, in Google Drive, the option "everyone with the link can view" is selected under the "Share" options).

Roles:
-Lecturer: Will provide the main lecture for the session and can also provide the recap on the previous session. Alternatively, you can choose a second lecturer to provide the recap from the previous session. -Video Conferencing Facilitator: Recommend that these persons be separate from the facilitators for the small groups. They will assign participants to the break out rooms, move Small Group Facilitators between the breakout rooms, and keep time for the breakout sessions. Also recommend that these persons have access to the handout and scenario script links to send out in the video conferencing chat. -Small Group Facilitators: Will also likely include the lecturer. Depending on the number of small groups, these facilitators can stay with one group or be moved through different groups by the Video Conferencing Facilitator. These facilitators should be experienced educators who are prepared to answer questions regarding the topics presented in the lecture so that they can efficiently guide the scenarios and participate in the debriefs. The timing once the role-playing scenarios start is more fluid and is also dependent on how the learners work through the scenarios. Recommend communicating back to the Video Conferencing Facilitator to end the role-playing scenarios as groups finish to start the debriefing discussions.

Content Outline:
Lecture: -Discuss importance of setting expectations with junior learners (medical students, offservice residents, interns) and orienting them to the ED (and how it is different from other specialties' environments) -How to identify the level of the learner (briefly so you know what expectations to set for them) -Define the different steps of the one-minute preceptor -Note that the content provided here adds the additional two steps proposed by Kansas University Medical Center for a total of seven -Provide examples of the different steps of the One-Minute-Preceptor model Role-playing scenarios: -Groups can be 2 or 3 residents, with one playing the student, one playing the resident, and an optional observer (in the case of groups of 3) -The third case can be considered optional if the discussion of the first two scenarios runs long -Ideally, there should be enough scenarios to allow each participant to play each role at least once -Ensure that all participants have access to the handout to reference prior to the start of the role-playing scenarios -Provide scripts for the student and resident portions of each scenario via links -Recommend including these in the handout reference -If desired, can direct the observer (if present) to look at specific aspects of the interaction (eg, one thing that was done really well and one thing that could be improved). -Ensure the learners are providing their own thoughts and ideas -Avoid feeding them answers to questions too early -Ask "What" Questions -"What do you think is going on?" -"What antibiotics would you use to treat this infection?" -If present, try to determine the reason for reluctance displayed by the learner -Reluctance may be multifactorial (feel intimidated, passive learning style, etc.)

Step 3: Probe for Supporting Evidence
-Ask "Why" Questions -"Why would you order a complete metabolic panel on this patient?" -Ensures understanding of the medical topics being discussed

Step 4: Reinforce what was Done Well
-Provide SPECIFIC positive feedback -"You did a great job talking with that patient's family!" Step 5: Correct Mistakes -Don't be afraid to point out mistakes so that a learner may correct them in the future -Provide SPECIFIC and ACTIONABLE feedback to the learner -"Next time, I would try rehearsing the consult call before paging surgery." -In many circumstances, it is most appropriate to correct mistakes away from public view Following completion of this session, residents will be able to: -Recognize and define the aspects of effective feedback.
-Apply the aspects of effective feedback on shift while providing feedback to the medical students. -Describe the common pitfalls of feedback.

Pre-Session Preparation:
-It is strongly recommended that the session handouts and the scenario scripts are loaded into a GoogleDrive folder. These can then be linked to the scenario scripts into the bottom of the handouts for ease of access. -Ensure that sharing permissions are correct (eg, in Google Drive, the option, "everyone with the link can view," is selected under the "Share" options).

Roles:
-Lecturer: Will provide the main lecture for the session and can also provide the recap on the previous session. Alternatively, you can choose a second lecturer to provide the recap from the previous session. -Video Conferencing Facilitator: Recommend that this person be separate from the facilitators for the small groups. They will assign participants to the break out rooms, move Small Group Facilitators between the breakout rooms, and keep time for the breakout sessions. Also recommend that this person have access to the handout and scenario script links to send out in the video conferencing chat. -Small Group Facilitators: Will also likely include the lecturer. Depending on the number of small groups, these facilitators can stay with one group or be moved through different groups by the Video Conferencing Facilitator. These facilitators should be experienced educators who are prepared to answer questions regarding the topics presented in the lecture so that they can efficiently guide the scenarios and participate in the debriefs. The timing once the role-playing scenarios start is more fluid and is also dependent on how the learners work through the scenarios. Recommend communicating back to the Video Conferencing Facilitator to end the role-playing scenarios as groups finish to start the debriefing discussions. Role-playing scenarios: -Groups can be 2 or 3 residents, with one playing the student, one playing the resident, and an optional observer (in the case of groups of 3). -The third case can be considered optional if the discussion of the first two scenarios runs long. -Ideally, there should be enough scenarios to allow each participant to play each role at least once. -Ensure that all participants have access to the handout to reference prior to the start of the role-playing scenarios. -Provide scripts for the student and resident portions of each scenario via links.

DIDACTICS AND HANDS-ON CURRICULUM
-Recommend including these in the handout reference. -If desired, can direct the observer (if present) to look at specific aspects of the interaction (eg, one thing that was done really well and one thing that could be improved).  S: I think I did a pretty good job seeing a lot of patients today.
R: I agree. You saw a good volume and had some interesting cases. I think you did a great job of keeping a broad, but reasonable differential. It's important not to anchor on a patient's chief complaint in EM. Was there anything you felt like you could improve?
S: On the appendicitis patient, it took me a while to realize that we hadn't put in pain medication orders for them until the nurse came and talked to me about it.
R: Yeah, it can be frustrating when we forget things. When things get busy in the ED, we have to rely on good communication with our team so nothing gets missed. I always try and circle back with the patient's nurse after I've seen the patient to give them an idea of what we need to do next and what orders they may need. Maybe you could try that next shift and see if you feel like it makes things more efficient.
S: Yeah, that's a good idea. I'll try and work with the nurses more next time.
This is your last shift working with this student and you are concerned. Based on today's shift, you think the student has some significant deficits with patients with neurologic complaints. You are not sure if the student has ever been told this before and want to make sure that the student knows additional studying is needed. Specifically, you thought that the student lacked some compassion dealing with the patient with suspected psychogenic nonepileptic seizures, as well as a systemic approach to the history taking and you would like to offer some advice about how to improve on the next shift. The timing once the role-playing scenarios start is more fluid and is also dependent on how the learners work through the scenarios. Recommend communicating back to the Video Conferencing Facilitator to end the role-playing scenarios as groups finish to start the debriefing discussions.

Content Outline:
Lecture: -Reviews the following teaching tools -"What if?" Game -Aunt Minnie -SPIT -Activated Demonstration -Self-directed teaching tools -Teaching scripts/mini lectures -Post-it Pearls Role-playing scenarios: -Groups can be 2 or 3 residents, with one playing the student, one playing the resident, and an optional observer (in the case of groups of 3). -The third case can be considered optional if the discussion of the first two scenarios runs long. -Ideally, there should be enough scenarios to allow each participant to play each role at least once. -Ensure that all participants have access to the handout to reference prior to the start of the role-playing scenarios. -Provide scripts for the student and resident portions of each scenario via links.

DIDACTICS AND HANDS-ON CURRICULUM
-Recommend including these in the handout reference. -If desired, can direct the observer (if present) to look at specific aspects of the interaction (eg, one thing that was done really well and one thing that could be improved).

Debrief:
-Elicit emotional response from the case.
-Examples: What was difficult about choosing a teaching strategy on the spot? How does it feel to correct the "medical student" if they made mistakes? -Discuss which teaching strategy the resident chose and why.
-Discuss any difficulties that may have occurred with the chosen teaching strategy.
-Discuss how the scenario felt from the "medical student's" perspective.
-Examples: Did the strategy feel like an effective way to learn? If you required correction, how did it feel? -Address any knowledge gaps and correct assumptions. 2021 You are a third-year medical student on your second clinical rotation, and this is your fourth ED shift. The resident has sent you into a room to evaluate a 25-year-old female with a chief complaint of headache. You have seen a few patients on your shifts so far with similar presentations.
As you present this patient, feel free to express that you have seen a lot of patients like this before and you feel confident in treating this as a primary headache. As the resident leads you through the patient encounter discussion, please feel free to answer some questions incorrectly to give them a chance to practice correcting mistakes.